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ABSTRACT 

West Georgia College, a public, 4-year institution in 
the University System of Georgia, conducted an assessment of its 
institutional research function. The assessment design consisted of 
two distinct elements: an internal program assessment, and an 
assessment of institutional research as perceived by campus 
constituents. The assessment was undertaken as part of the activities 
surrounding the entire institution's review and reaffirmation of 
accreditation from the Commission on Colleges Southern Association of 
Colleges and Schools (SACS). The assessment design for institutional 
research functions was developed in compliance with SACS 
accreditation criteria as well as the institution's goal of 
determining how effective the office was in carrying out its mission. 
The office developed a survey asking for evaluation on external 
reporting, faculty studies, student studies, academic operations, 
institutional studies, institutional effectiveness, and planning and 
coordination support. Results indicated that for the most part 
institutional research studies were reaching the right people with 
the right information at the right time. For the internal program 
review, the office established a set of goals and objectives to guide 
the effort. The office developed its own four-part model and 
established and defined program outcomes and expected results. 
(Contains 19 references.) (JB) 
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ABSTRACT 

Discussions of assessment in higher education are commonplace However, 
dialogue concerning the assessment of the institutional research office and function has 
been scarce This paper examines the planning and methodology of the assessment of 
the institutional research function at West Georgia College, a public 4-year institution, part 
of the University System of Georgia. The assessment design consists of two distinct 
elements; an internal program assessment, and an assessment of institutional research 
as perceived by campus constituents. Recommendations on the design, coordination, 
and dissemination of assessment findings are addressed. 
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Introduction 

Assessment is one mean to the end of improving the performance of individuals 
and institutions. It facilitates progress toward goals and objectives, and is vital for 
determining both efficiency and effectiveness. Institutional assessment is a complex, large 
scale, and in many ways, very subjective venture. It has a long history and is 
accompanied by an extensive literature. Assessment of the institutional research function 
and office is a more complex and ambiguous endeavor than the evaluation of other, more 
objective functions, such as academic programs or personnel. However, much more is 
known about assessment theories and methods than is usually applied; the professional 
diagnosis and prescription is often past over for the "quick fix". One of the major 
assessment related problems facing practitioners today is the fact that little, if any, of the 
current assessment literature is practical and 'user friendly'. The literature is much too 
mechanical and unrealistic. Even the jargon and argot associated with assessment 
makes acceptance and use highly doubtful. Like most other institutional research offices 
in our state, we have only two professional staff and simply cannot temporarily 
discontinue or suspend our support function in order to spend a great deal of time on 
assessment. 

In order for assessment activities to be more worthwhile they must be pragmatic. 
Successful assessment is predicated on achieving a balance between the theoretical and 
the practical. In our assessment effort at West Georgia College, we strived to integrate 
assessment methodologies and prescriptions, without completely abandoning the sterile 
approach of theory, or the often volatile, humanistic approach. 



Because decisions affecting institutional quality must be made daily, a meticulous 
employment of institutionally-specific assessment and evaluation measures will improve 
decision-making and institutional effectiveness, as the public, state, and federal 
government increase their demands for accountability in postsecondary education, we 
must begin to conduct more meaningful assessments and evaluations as part of our on- 
going decision-making activity. 

If assessment is to be more than merely an academic exercise, it should begin with 
our making some basic assumptions about evaluation and about what will be assessed. 
Miller (1985) identified several broad premises on the process of assessment, which we 
adopted: (1) institutional evaluation and assessment will be an increasing part of higher 
education through the rest of the century; (2) assessment is already being done by 
someone, somehow; if public postsecondary institutions do not take it upon themselves 
to conduct vigorous periodic evaluation, likely it will be done for them; (3) the overall 
purpose of institutional assessment activity should be the improvement of the academic 
enterprise; (4) assessment can be done objectively and perceptively; (5) assessment 
quality .can be defined; and (6) institutional quality is a composite of inter-dependent 
elements. 
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Defining Assessment 

For our purposes, assessment is defined as the systematic basis for making 
inferences about the effectiveness of the institutional research function and office in 
providing data and information in support of institutional planning, policy formulation, and 
decision-making. Assessment processes include discussion of what should be assessed 
and how information will be used (Erwin, 1991; Marchese, 1987). Assessment activities 
should be tied directly to the institutional research purpose statement which should follow 
directly from the institutional mission statement (Erwin, 1991). 

Successful assessment programs are ultimately a product of strong upper-level 
administrative support, flexibility in the design of assessment goals and methods, initial 
emphasis on institutional improvement rather than accountability, and commitment of 
human and fiscal resources to support assessment activities. It is imperative that we 
clearly communicate the purpose of assessment and how assessment results will be 
used. Building commitment from institutional constituents is a principal element in a 
successful assessment endeavor. The institutional research office must be given ample 
time and opportunity to establish goals, objectives, and evaluation methods in order to 
ensure meaningful assessment. The assessment process must be viewed as open and 
must at all times encourage wide-scale participation. If followed, these steps will minimize 
resistance to the assessment plan. 
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The Purpose of Assessing the Institutional Research Function and Office 

An ongoing cycle of assessment of the institutional research office and function can 
help validate institutional effectiveness. When undertaking the assessment of the 
institutional research function and office, be especially careful not to confuse assessment 
and planning. Assessment should identify the degree to which institutional research 
activities exist "campuswide" and seek to improve the effectiveness of those activities. 

The institutional research function is the activity of research conducted within an 
institution of higher education to provide information in support of institutional planning, 
policy formulation and decision making. Institutional research activities conducted 
campuswide can occur within units, schools, departments, and even committees. Such 
activities are conducted formally and informally, with specific objectives as well as simple 
'fact finding' in mind. 

The institutional research office is the organizational entity in which institutional 
research activities are the most highly centralized and concentrated. The responsibilities 
of an institutional research office frequently include carefully planning and conducting 
enrollment projections and studies, admissions and retention studies, institutional cost 
analyses, student, faculty, administrative and academic program studies, and producing 
formal written reports. The institutional research office provides such data and information 
to be used to plan and evaluate the institution's success in accompishing its' mission. 
The office also engages in continual study of institutional purpose, policies, procedures, 
and programs. 
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The primary purpose of our assessment activities was not to develop and 
implement an assessment model to satisfy our regional accrediting agency's assessment 
mandate as expressed in their Criteria. It was, instead, to provide general assistance in 
the development of acceptable assessment strategies and methods that would allow us 
to gather evidence useful for the improvement of the institutional research function and 
office. 

During 1992 and 1993, West Georgia College was deeply involved in assessment 
activities, preparing for the review and reaffirmation of our accreditation from the 
Commission on Colleges Southern Association of Colleges and Schools. Like our sister 
institutions, we evaluated our institutional mission, purpose and effectiveness, our 
educational programs, student support services, and administrative processes. In 
support of these activities, we prepared ourselves to examine our office and function of 
institutional research. The institutional research office on our campus is unique compared 
to most other public institutions. The office is housed in the Division of Budget and 
Research Services, which is primarily responsible for all planning activity, institutional 
research, sponsored operations, and budget. Although the integration of these units is 
preeminent to the overall efficiency of the Division, separating these functions in order to 
assess one of them as an individual entity becomes difficult at best. It is for this purpose 
that a pragmatic approach to assessment was not only preferred, but imperative. 
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Our Assessment Model 

The assessment of the institutional research function and office should be part of, 
not apart from, the institution's overall assessment program. It should also be tied to, but 
separate from, the program planning process. An institutional research office's purpose 
statement should tie directly to the institution's mission statement. Examination of the 
unit's purpose statement should result in a planning process identifying how the 
processes will be carried out and an assessment process that will determine how well the 
purpose has been carried out. 

Institutional research officers shouid be in the business of assessment. They must 
provide active leadership from within the institution to set the proper tone for assessment 
activities, be they administrative or academic in nature. Persuasive leadership is vital to 
the success of any planned change initiative 

Just as the structure of the institutional research office and function will differ 
among institutions, we strived to develop an assessment model that would allow sufficient 
freedom for any institution to customize its assessment process. Our assessment 
initiative made careful consideration of sampling issues, times for collecting data and 
information, use of technology, direct and indirect costs, maintaining the reliability and 
validity to the data collection techniques, the analysis of assessment results, and the 
reporting of assessment information. Analysis is the process of aggregating responses 
to various assessment stimuli and of describing the overall characteristics, patterns, and 
parts of these data to obtain answers to important questions concerning effectiveness. 
Interpreting is describing what the analyses mean and their implications. We followed 



guidelines (Brinkerhoff, Brethower, Hluchkyj, and Nowakowski, 1983) for the process of 
data analysis ana interpretation: (1) Review the questions being addressed. What does 
one seek to know from the assessment study? (2) Prepare the basic descriptive analyses, 
(3) Implement the selected analytical model for the available data, (4) Given the 
limitations of the model, analyze the basic issues, trends and relationships among the 
measures of interest, and (5) Noting the limitations of the analyses, interpret the findings. 

Erwin (1991) states that the purpose of any model for analyzing assessment 
information is to document the contribution, or lack thereof, of a particular program. 
Other related research (Cross and Angelo, 1988; Pascarella, 1987) states that assessment 
should be more concerned with the effectiveness of programs and experiences, and that 
the major value of assessment is to improve quality and effectiveness. 

When we began our assessment process we were sure of only two things; (1) we 
needed to assess the institutional research function in compliance with SACS 
accreditation criteria, and (2) we needed to examine exactly how effective we were in 
carrying out our office's mission. It was not the why , or the what of assessment that we 
found to be particular challenging, but the how . The process of institutional assessment 
is as important as the product. c or us, an appropriate program review would require two 
separate assessments; first an assessment of the Institutional Research Office (an internal 
program review), and secondly, an evaluation of the institutional research function, as 
perceived by our constituents across the campus. To facilitate the assessment of the 
institutional research function, we developed a survey instrument loosely based on 
Nichols (1989), asking the Executive Council, Deans and Directors, Department Chairs, 
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as well as other units and individuals who request, institutional research data on an ad hoc 
basis, to evaluate us on seven distinct criteria measures: external reporting; faculty 
studies; student studies; academic operations; institutional studies; institutional 
effectiveness; and planning and coordination support. 

Respondents were asked to identify their level of need for the stated institutional 
research criteria and the percentage of that need which was met by the Office of 
Institutional Research. Although ambitious, the instrument did not yield data that was as 
reliable as we had initially anticipated. Based on the mission of the Office of Institutional 
Research, our primary responsibility is to provide information to support the executive 
decision-making function of the institution. With that in mind, many of the criteria variables 
included in our instrument did not have meaningful significance for all of the constituent 
target groups and therefore the groups skewed responses often restricted objective 
analysis of certain variables. However, the data we collected did indicate that the 
variables bearing high levels of need for the Executive Council, and the Deans and 
Directors were being sufficiently and consistently met. 

We found that for the most part, the institutional research studies that we prepare 
on a regular basis are reaching the right people, and providing them with the right 
information at the right time. Even on items evaluating our contribution to overall 
institutional effectiveness and planning support, we found that an overwhelming majority 
of respondents believe that the Office of Institutional Research is effective in supporting 
the needs of those particular activities. 
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Another objective of our assessment activity was to conduct an internal program 
review, addressing several issues in the unit's history and mission, it's contribution to 
institutional planning, decision-making, and policy formulation functions. We wanted to 
address issues of organizational structure, including personnel and reporting structure, 
the institutional research budget, and our decision-making process of determining which 
projects are approved, how they are prioritized, and who pays for them. Routine and 
periodic institutional research activities, as well as infrequent requests for data were 
addressed to determine the patterns of the request source, information generated, and 
the information's target audience. The final component in our internal program review 
was to address issues surrounding unit domain of responsibility, resource allocation, 
equipment needs, and plans and projects to be undertaken in the near future. 

To assist the assessment endeavor, we established a set of goals and objectives 
that would guide our efforts. In our case, the goals also served as the assessment 
outcomes; having an identifiable record of relevant institutional research assessment 
activity that would address SACS criteria, and a unit plan for fiscal year 1992-93. To 
reinforce our two primary goals we established sets of objectives for each goal by 
determining appropriate assessment criteria (institutional support, administrative 
leadership, institutional self improvement, financial support, and external relations); the 
development of a survey instrument to assess the institutional research function; to 
administer the survey and analyze the results; and to conduct and internal evaluation of 
the institutional research office. 
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To facilitate the assessment of the institutional research office, we developed our 
own model, which consisted of four distinct elements; (1) an examination of the 
institutional research purpose statement, (2) a description of the major characteristics nd 
activities of the institutional research office, (3) identification of the components of our 
assessment model, and finally (4) reporting our assessment findings. In examining our 
purpose staten ant, we attempted to answer the questions, "What are the primary 
purposes of the Institutional Research office and function at our institution?". 
Furthermore, "How are these related to our administrative division's mission statement, 
and that of the institution?". Finally, "How is what we've done during the past year relate 
to each of these purpose statements?" 

In describing the major characteristics and activities of the office, we examined the 
human, physical, and fiscal resources available, the relationship of the Institutional 
Research office to the institution, and the office's primary responsibilities. In assessing 
human, physical, and fiscal resources, we looked at staff size, credentials and 
professional development activities, data processing equipment and support, clerical 
support services, and funding sources. We addressed issues of resource limitations 
where warranted. We attempted to stress that information management depends to an 
extent on the immediate availability and future acquisition of sophisticated software and 
hardware. In examining the relationship of the Institutional Research office to the 
institution we briefly discussed the locus of administrative control over the office and the 
real and perceived major lines of interaction with various institutional research 
constituents. Specific charges to the institutional research unit were determined by our 
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placement in the organizational hierarchy. Reporting to the Chief Administrative Officer 
for Budget and Planning, we are more heavily involved in support of the executive 
decision making function than support of academic affairs or student services. We 
assessed the balance between the centralized and decentralized practice of institutional 
research across the campus community. Lastly we listed all of the major areas of 
responsibility in the Institutional Research office, both routine and ad hoc, and answered 
the following questions: (1) What is being done?", (2) Who are we Serving?", and (3) What 
exactly are they trying to accomplish?", restated we set out to determine if Institutional 
Research is accomplishing its stated and broader missions. In this section we discussed 
the intended impact and effectiveness of our activities on certain functional areas of the 
institution. 

The next step involved the establishment and definition of program outcomes and 
expected results. It was important that the outcomes identified for the program 
assessment be consistent with the philosophy of the institutional research function and 
the overall mission of the institution. Each college and university must establish its own 
assessment program that is sensitive to the unique characteristics and qualities of the 
institution. Specific outcomes must be written and agreed to by all institutional research 
constituents. It is important that what is to be assessed, be stated as clearly as possible 
before any assessment methods is planned or data collected (Erwin, 1991). 

The outcomes of the institutional research function are often difficult to define and 
assess. They are dynamic and require constant review and revision. Institutional 
researchers should not attempt to 'reinvent the wheel'. They should consult with 
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professional organizations such as AIR, 3CUP, AAHE, as well as other colleagues for 
already existing examples of outcomes. Improving effectiveness in attaining these 
outcomes may require manipulation of the campus environment. Often referred to as 
'inputs', the campus environment can be described as a set of functions, comprising the 
programs, services, provisions, resources, and methods whereby purposes and expected 
outcomes are achieved (Dressel and Associates, 1961). These inputs are the resources 
of the institution and the characteristics which help offices and units reach prescribed 
outcomes. The campus environment includes instructional, social, fiscal, organizational, 
personal, and physical influences (Micek and Arney, 1974). 

We realized early on that the purpose of assessment is not to show cause and 
effect relationships. Thus our assessment focused on outcomes as indicators of the 
impact and effect of given programs and services as intended by both those responsible 
for the activity and function. 

Most importantly, do not assess only that which is easy to assess; trivial outcomes 
may impart false meaning. Strive to define outcomes that are central to the program or 
service under examination. Stress those outcomes that are truly significant. The 
assessment process can be refined; however, the consequences of defining a program 
around minimal outcomes might be that standards are lowered over time. The primary 
purpose for assessment is the improvement of quality of higher education. 
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Reporting Assessment Findings 



It was important that careful consideration be given to how to report the results and 
conclusions derived from the interpretations of our assessment information. The mode 
of presentation can b© x progress reports, a final report, a technical report, a summary, or 
multimedia presentation. Individual institutions customarily have their own preferred report 
format, but our assessment report included the following sections: 

1. Unit Purpose Statement. As included in the annual 
assessment document because it establishes the foundation upon which the 
assessment plan was built. 

2. Identify Outcomes and Methods. This included the core functions, 
each with at least one outcome, each of which should include at least one 
outcome statement accompanied by one or more expected result, each with 
at least one assessment method. 



3. Performing the Assessment. This section carefully detailed the 
implementation of assessment methods, the methods and techniques of 
obtaining data from the chosen assessment method(s), and the compiling 
of assessment data for analysis and reporting. 

4. Analysis of Results. This section detailed the Jtilization of our analysis 
techniques which provided the best and most thorough insights. We 
asked such questions as; Were expected outcomes obtained? What are 
the program and service delivery strengths and weaknesses? We focused 
on obtained results and how they varied from the expected results. We 
designed institution-specific strategies to implement improvement to the 
institutional research function and office. The most relevant comparisons 
of assessment data were internal and addressed the question "Are we doing 
better now, than we did before?" 

5. Conclusions/Future Plans for Improvement. We explained our 
conclusions and how the data would be used to bring about appropriate 
change to improve the effectiveness of the institutional research function and 
office. 



6. Assessment Evaluation. How well did the assessment program work? 
What adjustments need to be made in order to improve your assessment 
methods? 
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Conclusion 

Few assessment endeavors succeed without well conceived planning. Planning 
emerges in our assessment models and flowcharts, and is instrumental in the 
development of assessment goals and objectives. The planning function, like institutional 
research, takes place in every corner of the academic enterprise. Rarely can a 
conversation take place between two or more administrators without the words "planning" 
or "assessment" emerging. As we began to envelop our assessment plan, it was 
particularly helpful to keep in mind several of Stumer's planning principles, as they refer 
to assessment: planning involves deliberately designing actions that will lead to one's 
goals and objectives; it involves a systems approach, an attempt to understand and a 
perspective that affects the entire institution; it is dependent on collaboration and wide 
participation; an organization/unit is not easy to change, nor is it easy to plot or quantify 
the changes while they are in progress; uncertainty and ambiguity are a way of life; 
planning is concerned with creating realistic images of the future; and planning is an 
approach to management, a process by which one learns more about the 
organization/unit and slowly but continually plans and implement actions to bring about 
the images desired. 

As we learned from the process of assessing our institutional research efforts, the 
assessment process should be flexible and permit an assorted array of approaches; it 
should be attuned to current institutional priorities and utilize recent institutional research 
studies. Assessment should involve only as many individuals as is absolutely necessary 
(to avoid the 'Group Think' trap). It should have enough breadth and depth to review 
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accountability and should focus attention on the results of the program assessment. 
Assessment must use the information and results from the evaluation to improve the 
program functioning. Lastly, assessment should foster on-going self-study and planning. 

The way activities are conducted - the very process- is most important to eventual 
success or failure. Incorporated in the process is careful attention to composition of the 
organization/unit, systematic study design, and openness to discussion and comment 
from the unit's constituencies. It is much better to touch bases than mend fences. 
However, just as excessive bureaucratic control and secrecy surrounding assessment can 
diminish credibility and acceptance, unreasonable participation and collaboration can 
grind assessment to a standstill. 

Institutional assessment activities should use objective data where possible and 
applicable, but yet make no apologies for using subjective data. Objective data is critical, 
yet considerable variation exists in the availability and quality of such data. The absence 
of objective data should stimulate those responsible for assessment to develop their own 
survey instruments and guidelines as bases for decision making. Solid foundations for 
decision making can be developed by using whatever reliable hard data is available along 
with experience, judgement, and common sense. 

Assessment activities should be action oriented, with strategies for translating 
recommendations into institutional action. The administration would be remiss to label 
assessment activities as "filed, forgotten" considering the staff time and dollars invested 
in the assessment and the importance of keeping the institution abreast or ahead of its 
problems. 
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We discovered that our own assessment effort had succumbed to a number of 
problems that have been prominent in the assessment literature. A past study by the 
AASCU raised a number of significant cautions with respect to assessment procedures 
from which our assessment could have benefitted had we been more conscious: 
avoiding the mass dissemination of assessment findings as they may be distorted and 
used by other individuals and/or departments for self-serving purposes; avoiding over- 
stressing individual items apart from the context of the entire evaluation instrument; 
avoiding assigning the same value to different evaluations - examine the background of 
each respondent; and to avoid making final recommendations based on evaluation 
material that only represents a part of the total picture. 

It is really too premature to determine the success or failure of our assessment of 
activity and function of the Office of Institutional Research here at West Georgia College. 
Only time and experience will determine the outcome of our evaluation and the strategy 
for actions prompted by our study. We plan to repeat the assessment process during 
the Fall of 1994. Hopefully, the next evaluation effort will yield a clearer and more concise 
picture of our office, devoid of the pitfalls uncovered this time around. 

In closing, individuals are the ones that drive the organization, it's programs, and 
assessment and evaluations. The process of developing, implementing, and managing 
assessment activity is a human problem. The sensitivities and needs of individuals are 
genuine and must be considered when developing and implementing assessment, but a 
progressive and dynamic college or university is maintained by accentuating the positive 
and moving on. 

20 
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